Skip to content

fix(cdc): mark all leaves dirty when a TRUNCATE is decoded#145

Open
mason-sharp wants to merge 2 commits into
mainfrom
fix/ACE/mtree-truncate
Open

fix(cdc): mark all leaves dirty when a TRUNCATE is decoded#145
mason-sharp wants to merge 2 commits into
mainfrom
fix/ACE/mtree-truncate

Conversation

@mason-sharp

Copy link
Copy Markdown
Member

The CDC drain's message switch only handled Insert/Update/Delete, so the TruncateMessage pgoutput publishes for a TRUNCATE (the publication is created with the default publish list, which includes truncate) was silently dropped. No leaves were dirtied, the truncated node's Merkle tree kept its pre-truncate hashes, and mtree table-diff reported the nodes as in sync.

A truncate carries only relation OIDs -- no per-row changes -- so per-PK tracking is impossible. React with the adaptive drain's bulk path instead: mark every leaf of each truncated relation dirty (cascades list all published relations) so the next tree update rehashes from the live, now-empty table. Buffered UPDATEs for the table are purged as at escalation time; a failed mark is terminal so a stale tree is never reported current. Applies to both bounded drains (table-diff/update) and continuous listen, which share the decode switch.

The integration test reproduces the field report: identical nodes, clean diff, TRUNCATE on n2, then a second diff must surface every row instead of "Merkle trees are identical".

@mason-sharp mason-sharp requested a review from rasifr July 10, 2026 20:29
@coderabbitai

coderabbitai Bot commented Jul 10, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Changes

TRUNCATE Merkle synchronization

Layer / File(s) Summary
Process logical replication truncates
internal/infra/cdc/listen.go
TRUNCATE messages validate relation OIDs, mark Merkle leaves dirty, purge buffered CDC changes, optionally record escalation, and stop processing on errors.
Build the clean Merkle baseline
tests/integration/mtree_truncate_test.go
The test creates and cleans up a shared table, seeds identical rows on both nodes, and verifies an initially clean diff.
Validate post-truncate divergence
tests/integration/mtree_truncate_test.go
The test truncates only N2 in repair mode and verifies that the subsequent diff reports the seeded rows only on N1.

Poem

A rabbit watched the truncate stream,
Then marked each leaf for a fresh green dream.
N2 went bare, while N1 stayed bright,
The diff found every row just right.
Hop, hop—CDC now keeps things clean!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: handling decoded TRUNCATE messages by marking all leaves dirty.
Description check ✅ Passed The description is directly related to the PR and explains the TRUNCATE handling, purge behavior, and integration test.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/ACE/mtree-truncate

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codacy-production

codacy-production Bot commented Jul 10, 2026

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 15 complexity · 3 duplication

Metric Results
Complexity 15
Duplication 3

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@mason-sharp mason-sharp force-pushed the fix/ACE/mtree-truncate branch from 91c1f4d to 2e79ea7 Compare July 13, 2026 14:20
The CDC drain's message switch only handled Insert/Update/Delete, so the
TruncateMessage pgoutput publishes for a TRUNCATE (the publication is
created with the default publish list, which includes truncate) was
silently dropped. No leaves were dirtied, the truncated node's Merkle
tree kept its pre-truncate hashes, and mtree table-diff reported the
nodes as in sync.

A truncate carries only relation OIDs -- no per-row changes -- so per-PK
tracking is impossible. React with the adaptive drain's bulk path
instead: mark every leaf of each truncated relation dirty (cascades list
all published relations) so the next tree update rehashes from the live,
now-empty table. Buffered UPDATEs for the table are purged as at
escalation time; a failed mark is terminal so a stale tree is never
reported current. Applies to both bounded drains (table-diff/update) and
continuous listen, which share the decode switch.

The integration test reproduces the field report: identical nodes, clean
diff, TRUNCATE on n2, then a second diff must surface every row instead
of "Merkle trees are identical".
…cking

markTableTruncated already leaves every leaf dirty, which is exactly the
adaptive drain's escalated state. Record it in the escalator so UPDATEs
landing on a re-populated table later in the same bounded drain skip
per-PK tracking instead of re-running the threshold count and a redundant
mark-all-dirty. Guarded by esc != nil (nil in continuous mode);
INSERT/DELETE split-merge tracking is unaffected.
@mason-sharp mason-sharp force-pushed the fix/ACE/mtree-truncate branch from 2e79ea7 to 49da06a Compare July 14, 2026 00:58

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/integration/mtree_truncate_test.go (1)

106-110: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Log the failure of os.Remove during cleanup.

Although this is a best-effort cleanup, explicitly handling and logging potential errors from os.Remove aligns with the convention used elsewhere in the test (like MtreeTeardown) and ensures failures don't silently go unnoticed. Based on learnings, cleanup callbacks in integration tests should log expected failures with t.Logf rather than silently dropping them.

(Note: Ignoring the error from filepath.Glob is safe here since it only returns an error for malformed patterns, and the hardcoded pattern is valid.)

♻️ Proposed refactor
 		files, _ := filepath.Glob("*_diffs-*.json")
 		for _, f := range files {
-			os.Remove(f)
+			if err := os.Remove(f); err != nil {
+				t.Logf("Warning: failed to remove diff file %s: %v", f, err)
+			}
 		}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/mtree_truncate_test.go` around lines 106 - 110, Handle the
error returned by os.Remove in the cleanup loop and log any failure with t.Logf,
matching the existing MtreeTeardown cleanup convention while preserving
best-effort cleanup. Keep the filepath.Glob error handling unchanged.

Source: Learnings

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/integration/mtree_truncate_test.go`:
- Around line 106-110: Handle the error returned by os.Remove in the cleanup
loop and log any failure with t.Logf, matching the existing MtreeTeardown
cleanup convention while preserving best-effort cleanup. Keep the filepath.Glob
error handling unchanged.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 4ff4d915-3f6d-49b8-a961-8e136e0a3252

📥 Commits

Reviewing files that changed from the base of the PR and between 774645d and 49da06a.

📒 Files selected for processing (2)
  • internal/infra/cdc/listen.go
  • tests/integration/mtree_truncate_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • internal/infra/cdc/listen.go

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants